Image display system, image display method, movable object including the image display system, and non-transitory computer-readable medium

ABSTRACT

An image display system includes a display unit displaying an image, a projection unit projecting in a target space a virtual image corresponding to the image with an output light of the display unit, a body unit provided thereto the display unit and the projection unit, and an image producing unit including a first correction unit and a second correction unit. The first correction unit performs a first correction processing of correcting, based on a first orientation signal indicative of a first orientation change of the body unit, a display position of the virtual image in the target space. The second correction unit performs a second correction processing of correcting, based on a second orientation signal indicative of a second orientation change of the body unit which is faster than the first orientation change, the display position of the virtual image in the target space.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 17/230,556, filed Apr. 14, 2021, which is a continuation ofU.S. patent application Ser. No. 16/354,820 (now U.S. Pat. No.11,004,424), filed Mar. 15, 2019, which claims the benefit of JapanesePatent Application No. 2018-053557, filed Mar. 20, 2018. The disclosureof each of the above-noted applications is expressly incorporated hereinby reference in its entirety.

TECHNICAL FIELD

The present disclosure relates generally to image display systems, imagedisplay methods, movable objects including the image display systems andnon-transitory computer-readable mediums, and more particularly, to animage display system for projecting a virtual image in a target space,an image display method, a movable object including the image displaysystem, and a non-transitory computer-readable medium.

BACKGROUND ART

Document 1 (JP2017-13590A) discloses a head-up display device which isan example of an image display system projecting a virtual image in atarget space. This head-up display device includes an indicator foroutputting a display light, and is configured to emit the display lightto a front window of a vehicle to allow a user to visually perceive avirtual image of a display image of the indicator as if it issuperimposed on a real scene in front of the vehicle.

The head-up display device includes a vehicle vibration input unit and adisplay control unit. The vehicle vibration input unit is configured toreceive vibration information of the vehicle (i.e., orientationinformation of the vehicle). The display control unit is configured tocorrect the display image based on the vibration information to correcta display position of the virtual image. Specifically, the displaycontrol unit draws the display image on a first layer, and then moves adisplay position of the display image drawn on the first layer in adirection so as to compensate the vibration of the vehicle. The firstlayer is a virtual plane and is defined in association with a displaysurface of the indicator. As a result, the positional relation betweenthe display position of the virtual image and a target object existingin the real scene can be maintained regardless of the vibration of thevehicle.

In the above head-up display device, the display position of the displayimage is corrected based on the vibration information during a drawingprocessing of drawing the display image. In this configuration, thedisplay unit is subject to have a display delay (for example, a delayabout several frames). Furthermore, when the correction includes adisplacement of the display position of the display image, the displayimage may be partially out of range of the display surface by thedisplacement (to cause an image defect). In order to prevent such animage defect, a drawable area of the display image in which the displayimage is allowed to be drawn should be restricted to a comparativelynarrow area.

SUMMARY

The present disclosure provides an image display system, an imagedisplay method, a movable object including the image display system, anda non-transitory computer-readable medium, which are capable of reducinga display delay of a display image and a drawable area of the displayimage of which is less likely to be restricted.

An image display system according to an aspect of the present disclosureincludes an image producing unit, a display unit, a projection unit, anda body unit. The image producing unit is configured to produce an image.The display unit is configured to display the image produced by theimage producing unit. The projection unit is configured to project, in atarget space, a virtual image corresponding to the image with an outputlight of the display unit. The display unit and the projection unit areprovided to the body unit. The image producing unit includes a firstcorrection unit and a second correction unit. The first correction unitis configured to perform a first correction processing of correcting,based on a first orientation signal, a display position of the virtualimage in the target space. The first orientation signal is indicative ofa first orientation change of the body unit. The second correction unitis configured to perform a second correction processing of correcting,based on a second orientation signal, the display position of thevirtual image in the target space. The second orientation signal isindicative of a second orientation change of the body unit. A changerate of the second orientation change is faster than that of the firstorientation change. The second correction unit is configured to performthe second correction processing at a timing different from a timing atwhich the first correction unit performs the first correctionprocessing.

A movable object according to an aspect of the present disclosureincludes the image display system and a movable object body. The imagedisplay system is installed in the movable object body.

An image display method according to an aspect of the present disclosureis an image display method employing an image display system including adisplay unit, a projection unit, and a body unit. The image display unitis configured to display an image. The projection unit is configured toproject, in a target space, a virtual image corresponding to the imagewith an output light of the display unit. The display unit and theprojection unit are provided to the body unit. The image display methodincludes an image producing processing of producing the image displayedon the display unit. The image producing processing includes a firstcorrection processing and a second correction processing. The firstcorrection processing is a processing of correcting, based on a firstorientation signal, a display position of the virtual image in thetarget space. The first orientation signal is indicative of a firstorientation change of the body unit. The second correction processing isa processing of correcting, based on a second orientation signal, thedisplay position of the virtual image in the target space. The secondorientation signal is indicative of a second orientation change of thebody unit. A change rate of the second orientation change is faster thanthat of the first orientation change. The image display method performsthe first correction processing and the second correction processing atdifferent timings.

A non-transitory computer-readable medium according to an aspect of thepresent disclosure records a computer program for instructing a computersystem to execute the above image display method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a conceptual view illustrating a configuration of an imagedisplay system of one embodiment.

FIG. 2 is a conceptual view of an automobile including the image displaysystem.

FIG. 3 is a conceptual view of a field of view of a driver using thedisplay system.

FIG. 4 is a block diagram illustrating a configuration of an imagedisplay unit of the image display system.

FIG. 5A is a conceptual view illustrating a rendering method of an imagein a three-dimensional virtual space in a case where no orientationchange occurs in a body unit.

FIG. 5B is a conceptual view of a field of view of a driver in the casewhere no orientation change occurs in the body unit.

FIG. 6A is a conceptual view illustrating a rendering method of an imagein a three-dimensional virtual space in a case where a first orientationchange occurs in the body unit.

FIG. 6B is a conceptual view of a field of view of a driver in the casewhere the first orientation change occurs in the body unit.

FIG. 7 is a conceptual view illustrating a method for performing avibration correction processing and a distortion correction processing.

FIG. 8 shows a graph illustrating an example of a relation between achange amount (vibration amount) and a correction amount according tothe vibration correction processing.

FIG. 9 is a conceptual view illustrating a variation of a display unit.

DETAILED DESCRIPTION

(1) Overview

As shown in FIG. 1 and FIG. 2 , an image display system 10 according tothe present embodiment is a Head-Up Display (HUD), and may be installedin an automobile 100 which is an example of a movable object body, forexample. According to the present embodiment, a movable object isdefined as a whole object including the image display system 10 and theautomobile 100 (movable object body) in which the image display system10 is installed.

The image display system 10 is installed in an interior of theautomobile 100 to project an image onto a windshield 101 of theautomobile 100 from below. In an example shown in FIG. 2 , the imagedisplay system 10 is placed inside a dashboard 102 below the windshield101. When an image is projected from the image display system 10 ontothe windshield 101, a user (driver) 200 can visually perceive the imageprojected onto the windshield 101 as a virtual image 300 that is formedin a target space 400 in front of (outside) the automobile 100.

In this disclosure, a “virtual image” means an image which is formed bydiffused rays of light caused when imaging light emitted from the imagedisplay system 10 is diffused by a reflective member such as thewindshield 101 and appears as if a real object. The windshield 101 is atarget of projection onto which an image 700 is to be projected.

Therefore, as shown in FIG. 3 , the user 200 driving the automobile 100can see the virtual image 300 which is formed by the image displaysystem 10 and overlaps with a real scene spreading in front of theautomobile 100. Accordingly, the image display system 10 can presentvarious kinds of driving assist information in a form of the virtualimage 300 overlaid on the real scene to allow the user 200 to visuallyperceive the information. Examples of the driving assist informationinclude vehicle speed information, navigation information, pedestrianinformation, forward vehicle information, lane departure information,and vehicle condition information. Accordingly, when the user 200 trainshis or her eyes on a space in front of the windshield 101, the user 200can visually obtain the driving assist information by slight movement ofa line of his or her sight.

For example, the virtual image 300 displayed in the target space 400 canpresent the navigation information indicating a traveling direction ofthe automobile 100, and can be displayed as an arrow indicating aright-turn or left-turn as if it is drawn on a road surface 600. Such akind of virtual image 300 is displayed by use of the Augmented Reality(AR) techniques, and is overlaid on a specific position in the realscene (road surface 600, building, surrounding vehicle, pedestrian, andthe like) in a field of view of the user 200.

The virtual image 300 displayed in the target space 400 is presentwithin an imaginary surface 501 across an optical axis 500 of the imagedisplay system 10. The optical axis 500 extends along a road surface 600in front of the automobile 100 in the target space 400 in front of theautomobile 100. The imaginary surface 501 where the virtual image 300 iscreated is almost perpendicular to the road surface 600. For example,when the road surface 600 is a horizontal surface, the virtual image 310may be seen as if it extends along a vertical surface.

As shown in FIG. 1 , the image display system 10 includes an imagedisplay unit 2, a projection unit 3, and a body unit 1. The imagedisplay unit 2 includes a display surface 20 a, and is configured todisplay the image 700 on the display surface 20 a. The projection unit 3is configured to perform a projection processing of projecting thevirtual image 300 (see FIG. 2 ) corresponding to the image 700 towardthe target space 400 (see FIG. 2 ) with the output light of the imagedisplay unit 2. The image display unit 2 and the projection unit 3 areprovided to the body unit 1.

The body unit 1 is installed in the automobile 100. When an orientation(posture) of the automobile 100 changes due to, for example, a conditionof the road surface 600, acceleration/deceleration of the automobile100, and the like, an orientation of the body unit 1 also occursaccording to the orientation change of the automobile 100. Specifically,when the automobile 100 is decelerated, the automobile 100 inclinesforward, leading to a forward inclination of the body unit 1. When theautomobile 100 is accelerated, the automobile 100 inclines rearward,leading to a backward inclination of the body unit 1. Change in theorientation of the body unit 1 of the image display system 10 causes achange in the positional relation between the virtual image 300 and thereal scene (objects in the real scene). Accordingly, the orientationchange of the body unit 1 causes the virtual image 300 to be displayedat a position which is displaced (misaligned) from the specific positionoriginally desired to be displayed, in the real scene of the field ofview of the user 200, for example.

In view of such circumstances, as shown in FIG. 4 , the image displaysystem 10 includes a first correction unit 41 and a second correctionunit 42 configured to correct (adjust) the display position of thevirtual image 300 in the target space 400 in response to the orientationchange of the body unit 1 so that the virtual image 300 can be displayedat the specific position originally desired in the real scene. The firstcorrection unit (first adjusting unit) 41 is configured to perform afirst correction processing (first adjusting processing) of correcting(adjusting) the display position of the virtual image 300 in the targetspace 400 according to a first orientation change of the body unit 1.The second correction unit (second adjusting unit) 42 is configured toperform a second correction processing (second adjusting processing) ofcorrecting (adjusting) the display position of the virtual image 300 inthe target space 400 according to a second orientation change of thebody unit 1. The second correction unit 42 performs the secondcorrection processing at a timing different from a timing at which thefirst correction unit 41 performs the first correction processing. Achange rate of the second orientation change is faster than that of thefirst orientation change.

The first orientation change is a kind of orientation change of the bodyunit 1 of which change rate is comparatively slow. Specifically, thefirst orientation change is a vibration (orientation change) of whichfrequency is lower than a first frequency. In other words, the firstorientation change is a kind of orientation change of the body unit 1 ofwhich frequency of occurrence is smaller than a first threshold. In yetother words, the first orientation change is a kind of orientationchange of the body unit 1 that an orientation (posture) of the body unit1 after the orientation change is maintained for a certain length oftime. The first orientation change may occur by theacceleration/deceleration of the automobile 100, for example.

The second orientation change is a kind of orientation change of thebody unit 1 of which change rate is comparatively fast. Comparing thesecond orientation change and the first orientation change, the changerate of the second orientation change is faster than that of the firstorientation change. Specifically, the second orientation change is avibration (orientation change) of the body unit 1 of which frequency isequal to or higher than a second frequency. In other words, the secondorientation change is a kind of orientation change of the body unit 1 ofwhich frequency of occurrence is larger than or equal to a secondthreshold. In yet other words, the second orientation change is a kindof orientation change of the body unit 1 that the orientation (posture)of the body unit 1 changes with time. The second orientation change mayoccur, for example, by an up-down movement of the automobile 100 as aresult of the automobile 100 running on a bumpy road. The firstfrequency and the second frequency may be the same as or different fromeach other. The first threshold and the second threshold may be the sameas or different from each other, as well.

With this configuration, the display position of the virtual image 300can be corrected (adjusted) in response to the orientation change of thebody unit 1. As a result of which, even when the automobile 100 has thefirst orientation change or the second orientation change, the imagedisplay system 10 can display the virtual image 300 overlaid on thespecific position originally desired to be displayed in the real scenein the field of view of the user 200.

Furthermore, according to the image display system 10, the firstcorrection unit 41 and the second correction unit 42 work alternativelyaccording to the kinds of the orientation change of the automobile 100(first orientation change or second orientation change). The body unit 1possibly has various kinds of orientation change along with theorientation change of the automobile 100, but, according to the imagedisplay system 10, it is possible to correct the display position of thevirtual image 300 at an adequate processing stage among severalprocessing stages performed by the image display unit 2, depending onthe kind of the orientation change (whether it is the first orientationchange or the second orientation change) actually occurring in the bodyunit 1.

A correction processing of the display position of the virtual image 300addressing the second orientation change may be require to be performedin real time, and thus may be performed at a processing stage whichprocesses in a real-time basis, for example. A correction processing ofthe display position of the virtual image 300 addressing the firstorientation change may not be necessarily performed in real time, andthus may be performed at a processing stage in which a drawable area ofthe image 700 is less likely to be restricted by the correctionprocessing. Accordingly, it is possible to correct the display positionof the virtual image 300 according to the orientation change of the bodyunit 1, while a display delay of the virtual image 300 can be reducedand a drawable area of the image 700 corresponding to the virtual image300 is less likely to be restricted.

(2) Details

As shown in FIG. 1 , the image display system 10 includes the body unit1, the image display unit 2, the projection unit 3, a vibration sensor5, a low-pass filter 6, and a high-pass filter 7. The vibration sensor 5and the low-pass filter 6 cooperate to function as a first detectorconfigured to detect the first orientation change of the body unit 1 tooutput a first orientation signal indicative of the detected firstorientation change. The vibration sensor 5 and the high-pass filter 7cooperate to function as a second detector configured to detect thesecond orientation change of the body unit 1 to output a secondorientation signal indicative of the detected second orientation change.

The body unit 1 includes a housing, for example. The image display unit2, the projection unit 3, the vibration sensor 5, the low-pass filter 6,and the high-pass filter 7 are housed in (mounted on) the body unit 1.As shown in FIG. 2 , the body unit 1 is fixed to an interior of thedashboard 102 of the automobile 100. Alternatively, the body unit 1 mayinclude a frame or a plate member, in place of the housing. The imagedisplay unit 2 includes a display unit 20 and an image producing unit 21(described later). Out of the display unit 20 and the image producingunit 21, the image producing unit 21 may not be provided to the bodyunit 1.

The vibration sensor 5 includes a sensor configured to detector avibration acting on the body unit 1, and to detect the vibration of thebody unit 1 to detect the first orientation change and the secondorientation change of the body unit 1. The vibration on the body unit 1includes a first vibration having a frequency lower than the firstfrequency (corresponding to information on the first orientation change)and a second vibration having a frequency higher than or equal to thesecond frequency (corresponding to information on the second orientationchange). The body unit 1 is fixed to the automobile 100, and thus thevibration acting on the body unit 1 is substantially the same as thevibration acting on the automobile 100. That is, the vibration sensor 5is configured to detect the vibration acting on the automobile 100 andto thereby detect the vibration acting on the body unit 1. The vibrationsensor 5 is configured to detect a change in an angle of pitch of theautomobile 100 caused by the vibration on the automobile 100. Thevibration sensor 5 is configured to output a signal indicative of thedetected vibration, to the low-pass filter 6 and the high-pass filter 7.

Receiving the signal from the vibration sensor 5, the low-pass filter 6allows a signal component having a frequency lower than the firstfrequency (namely, a signal indicative of the first orientation change)to pass therethrough, and cuts off a signal component having a frequencyhigher than or equal to the first frequency. The low-pass filter 6outputs, as the first orientation signal, the signal (signal component)that passes through the low-pass filter 6. Receiving the signal from thevibration sensor 5, the high-pass filter 7 allows a signal componenthaving a frequency higher than or equal to the second frequency (namely,a signal indicative of the second orientation change) to passtherethrough, and cuts off a signal component having a frequency lowerthan the second frequency. The high-pass filter 7 outputs, as the secondorientation signal, the signal (signal component) that passes throughthe high-pass filter 7. The first orientation signal is indicative ofthe first orientation change of the body unit 1. The second orientationsignal is indicative of the second orientation change of the body unit1. The vibration sensor 5 may be a gyro-sensor (angular velocitysensor), for example. However, the vibration sensor 5 is not limited tothe gyro-sensor. The vibration sensor 5 may be an acceleration sensor,for example.

The image display unit 2 has the display surface 20 a, and is configuredto display the image 700 on the display surface 20 a to emit rays oflight constituting the displayed image 700 toward the projection unit 3.As shown in FIG. 1 , the image display unit 2 includes the display unit20 and the image producing unit 21. The display unit 20 is configured toperform a display processing of displaying the image 700 to emit therays of light constituting the displayed image 700 forward of thedisplay unit 20. The image producing unit 21 is configured to perform animage producing processing of producing the image 700 to be displayed onthe display unit 20. In the present embodiment, the image 700 includesan arrow for guiding a traveling route to a destination of theautomobile 100, which is displayed as if it is on a road surface for theautomobile 100. Hereinafter, the image 700 may be referred to as anarrow image 700. The virtual image 300 also includes an imagerepresenting an arrow (see FIG. 3 ).

Specifically, the image producing unit 21 is configured to acquire thenavigation information from a navigation device installed in theautomobile 100. The navigation information may include a traveling routeto the destination of the automobile 100, a distance from the automobile100 to a nearest intersection, and the like, for example. In an example,when determining that the automobile 100 reaches a location, which is acertain distance away from a nearest intersection in the travelingroute, based on the acquired navigation information, the image producingunit 21 produces the arrow image 700 for guiding the traveling directionin the intersection. In this instance, the arrow image 700 is displayedso that the virtual image 300 is displayed at a predetermined position(for example, a position where the arrow overlaps with the road in theintersection) 401 in the target space 400. The image producing unit 21then outputs the produced arrow image 700 to the display unit 20.Accordingly, the virtual image 300 corresponding to the arrow image 700is displayed at the predetermined position (for example, the positionwhere the arrow overlaps with the road in the intersection) 401 in thetarget space 400, provided that no orientation change occurs in the bodyunit 1.

The image producing unit 21 is configured to receive the firstorientation signal from the low-pass filter 6 and the second orientationsignal from the high-pass filter 7, respectively, to acquire orientationinformation about the first orientation change and the secondorientation change of the body unit 1. When the body unit 1 has anorientation change, the positional relation between the virtual image300 and the real scene (objects in the real scene) changes, leading to amisalignment (displacement) of the virtual image 300 with respect to aspecific position (for example, the intersection) originally desired tobe displayed in the target space 400 in the real scene. For example, thebackward inclination of the automobile 100 causes the line of the user's200 sight to move upward, and thus the real scene recognized by the user200 moves relatively downward. As a result, the virtual image 300 isdisplayed in the target space 400 at a position above the intersection.In view of this circumference, the image producing unit 21 corrects,based on the acquired orientation information, the display position ofthe image 700 in the display surface 20 a so that the display positionof the virtual image 300 in the target space 400 is displaced from thepredetermined position 401 to a position where the arrow overlaps withthe road in the intersection.

The projection unit 3 is configured to project the virtual image 300corresponding to the image 700 toward the target space 400 with anoutput light of the display surface 20 a of the image display unit 2. Inthe present embodiment, the image display system 10 functions as ahead-up display as described above, and the projection unit 3 isconfigured to project the image 700 onto the windshield 101 (see FIG. 2).

As shown in FIG. 1 , the projection unit 3 includes a first mirror 31and a second mirror 32. The first mirror 31 and the second mirror 32 aredisposed along an optical path of the output light of the image displayunit 2, in an order of the image display unit 2, the first mirror 31,and the second mirror 32. Specifically, the first mirror 31 is placedabove the display surface 20 a of the image display unit 2 to receivethe output light of the image display unit 2. The first mirror 31reflects the output light of the image display unit 2 toward the secondmirror 32. The second mirror 32 is placed at a position where the lightoutput from the image display unit 2 and reflected by the first mirror31 can reach (for example, at a position front and lower side withrespect to the first mirror 31). The second mirror 32 reflects upward(i.e., toward the windshield 101) the light output from the imagedisplay unit 2 and reflected by the first mirror 31. For example, thefirst mirror 31 may be a convex mirror. The second mirror 32 may be aconcave mirror.

With this configuration, the projection unit 3 can appropriatelyenlarged or minified the image 700 displayed on the display surface 20 aof the image display unit 2 to form a projection image projected ontothe windshield 101. Accordingly, the virtual image 300 is displayed inthe target space 400. In the field of view of the user 200 driving theautomobile 100, the virtual image 300 corresponding to the image 700 andprojected by the image display system 10 is present with the virtualimage 300 overlapped with the real scene spreading in front of theautomobile 100.

The image producing unit 21 and the display unit 20 will be explained indetail with reference to FIG. 4 .

As shown in FIG. 4 , the image producing unit 21 includes a drawing unit24 and a correction unit (adjustment unit) 25. The drawing unit 24 isconfigured to perform a drawing processing of drawing the image 700 (seeFIG. 1 ) to be displayed on the display unit 20. The drawing unit 24includes a drawing main unit 241 and a drawing buffer 242. The drawingmain unit 241 is configured to produce the image 700. The drawing buffer242 is configured to temporarily store the image 700 drawn by thedrawing main unit 241 and to output it to the correction unit 25. Thecorrection unit 25 is configured to perform a correction processing onthe image 700 drawn by the drawing unit 24 to perform variouscorrections, and to output to the display unit 20 the image 700 subjectto the correction processing.

In the present embodiment, the drawing main unit 241 draws the image(arrow image) 700 for guiding the traveling direction in theintersection, based on the navigation information acquired by the imageproducing unit 21, as described above.

The drawing main unit 241 is further configured to correct (adjust) theimage 700 based on the first orientation signal from the low-pass filter6 to compensate the misalignment of the virtual image 300 with respectto the real scene, which is caused by the first orientation change ofthe body unit 1. The correction processing (first correction processing)is performed on the image 700 during the drawing processing of the image700. Specifically, the drawing main unit 241 is configured to, whendetermining that the first orientation change occurs in the body unit 1based on the first orientation signal from the low-pass filter 6,correct the image 700 by drawing the image 700 so that the misalignmentof the virtual image 300 with respect to the real scene caused by thefirst orientation change of the body unit 1 is compensated. Accordingly,even when the first orientation change occurs in the body unit 1, thevirtual image 300 can be displayed in the target space 400 at thespecific position originally desired to be displayed in the real scene(for example, the position where the arrow overlaps with the road in theintersection) in the field of view of the user 200.

As described above, the drawing main unit 241 is configured to perform,on the image 700, the correction processing (first correctionprocessing) for correcting the display position of the virtual image 300based on the first orientation signal, in response to the firstorientation change of the body unit 1. In the present embodiment, thedrawing main unit 241 (namely, the drawing unit 24) serves as (includes)the first correction unit 41 configured to correct the display positionof the virtual image 300 in response to the first orientation change ofthe body unit 1.

Specifically, the drawing main unit 241 is configured to render, in athree-dimensional virtual space corresponding to the target space 400, athree-dimensional image corresponding to the image 700, and to projectthe rendered three-dimensional image onto a projection plane in thethree-dimensional virtual space to form a two-dimensional image servingas the image 700, and to thereby draw the image 700. The drawing mainunit 241 is further configured to correct the image while drawing theimage 700 by correcting a position and/or an angle of thethree-dimensional image while rendering the three-dimensional image inthe three-dimensional virtual space.

As described above, the correction processing of correcting the displayposition of the virtual image 300 addressing the first orientationchange of the body unit 1 (which is the correction processing notnecessarily to be performed in real time) is performed during thedrawing processing of the image 700. This can reduce a possibility thatthe image 700 is displayed on the display surface 20 a with the image700 partially out of range of the display surface 20 a (image defect).That is, this is addressed to reduce the possibility of the image defectrather than the real time drawability. As a result, the drawable area ofthe image 700 is less likely to be restricted.

As shown in FIG. 4 , the correction unit 25 includes an image correctionunit 251, an output buffer 252 (buffer), and a vibration distortioncorrection unit 253 (correction main unit).

The image correction unit 251 is configured to perform a colorcompensation processing on the image 700 output from the drawing buffer242 (i.e., on the image drawn by the drawing unit 24). The colorcompensation processing may include a correction processing of adjustingthe color of each picture unit (pixel) of the image 700. Specifically,the image correction unit 251 is configured to correct the color of eachpicture unit of the image 700 drawn by the drawing unit 24 so that thepicture unit has a desired color. For example, when there is a pictureunit whose color is defined as white but is tinged with red in the image700 drawn by the drawing unit 24, the color of the picture unit iscorrected to the white. The image correction unit 251 is configured tooutput, to the output buffer 252, the image 700 subject to the colorcompensation.

The output buffer 252 is configured to temporarily store the image 700output from the image correction unit 251, and to output the image 700to the vibration distortion correction unit 253.

The vibration distortion correction unit 253 is configured to perform avibration correction processing (second correction processing) and adistortion correction processing on the image 700 output from the outputbuffer 252 (i.e., on the image drawn by the drawing unit 24). Thevibration distortion correction unit 253 is configured to output, to thedisplay unit 20, the image 700 subject to the correction.

For the vibration correction processing (second correction processing),the vibration distortion correction unit 253 corrects, based on thesecond orientation signal from the high-pass filter, the image 700output from the output buffer 252 to compensate the misalignment of thevirtual image 300 with respect to the real scene, which is caused by thesecond orientation change. Specifically, the vibration distortioncorrection unit 253 is configured to, when determining that thevibration (namely, first orientation change) occurs in the body unit 1based on the second orientation signal from the high-pass filter 7,correct the image 700 output from the output buffer 252 so that themisalignment of the virtual image 300 with respect to the real scenecaused by the vibration of the body unit 1 is compensated. Specifically,in this correction processing, the image 700 is corrected so that theimage 700 to be displayed on the display surface 20 a is displacedtoward a desired direction (so that the misalignment of the virtualimage 300 with respect to the real scene caused by the vibration of thebody unit 1 is compensated). That is, the vibration distortioncorrection unit 253 is configured to perform, on the image 700, thecorrection processing (second correction processing) for correcting thedisplay position of the virtual image 300 based on the secondorientation signal. In the present embodiment, the vibration distortioncorrection unit 253 (i.e., the correction unit 25) serves as (includes)the second correction unit 42 configured to correct the display positionof the virtual image 300 in response to the second orientation change ofthe body unit 1.

With the vibration correction processing, even when the secondorientation change occurs in the body unit 1, the virtual image 300 canbe displayed in the target space 400 at the specific position originallydesired to be displayed in the real scene (for example, the positionwhere the arrow overlaps with the road in the intersection) in the fieldof view of the user 200. That is, the correction unit 25 serves as thesecond correction unit 42 configured to correct the display position ofthe virtual image 300 according to the second orientation change of thebody unit 1.

In the distortion correction processing, the vibration distortioncorrection unit 253 corrects the image 700 output from the output buffer252 so that the virtual image 300 projected onto the windshield 101 andreflected by the projection unit 3 has less distortion. In the presentembodiment, the distortion correction processing is performed after thevibration correction processing. The vibration correction processing andthe distortion correction processing are performed sequentially in thisorder.

As described above, the correction processing of correcting the displayposition of the virtual image 300 addressing the second orientationchange of the body unit 1 (which is the correction processing requiredto be performed in real time) is performed at a stage later than theoutput buffer 252 and before the display unit 20 (namely, at the laststage of the image producing processing). It is therefore possible toperform the correction processing of correcting the display position ofthe virtual image 300 addressing to the second orientation change of thebody unit 1 with a less display delay in the display unit 20.

The image producing processing of the image producing unit 21 forproducing the image 700 includes several steps which are sequentiallyperformed. The several steps correspond to steps of processing of: thedrawing main unit 241; the drawing buffer 242; the image correction unit251; the output buffer 252; and the vibration distortion correction unit253, respectively. The first correction unit 41 and the secondcorrection unit 42 perform the first correction processing and thesecond correction processing at different steps (a step corresponding tothe processing of the drawing main unit 241 and a step corresponding tothe processing of the vibration distortion correction unit 253,respectively) of the above several steps. That is, the first correctionunit 41 performs the first correction processing at the stepcorresponding to the processing of the drawing main unit 241. On theother hand, the second correction unit 42 performs the second correctionprocessing at the step corresponding to the processing of the vibrationdistortion correction unit 253. Since the first correction processingand the second correction processing are performed at different stepsfrom each other, the first correction processing and the secondcorrection processing can be performed at processing stages adequate fortheir properties (for example, whether the processing should beperformed in real time or not).

The drawing unit 24 and the correction unit 25 each include amicrocomputer including a Central Processing Unit (CPU) and a memory,for example. In other words, each of the drawing unit 24 and thecorrection unit 25 is realized by a computer (processor) including theCPU and the memory, and the computer, when the CPU executes a computerprogram stored in the memory, functions as the drawing unit 24 or thecorrection unit 25. The computer program for the drawing unit 24 isstored in the memory of the computer for the drawing unit 24 and thecomputer program for the correction unit 25 is stored in the memory ofthe computer for the correction unit 25, but at least part of them canbe provided through a telecommunication network such as the Internet ora non-transitory recording medium such as a memory card.

As shown in FIG. 4 , the display unit 20 includes a liquid crystal panel201, a light source device 202, and a display control unit 203. Theliquid crystal panel 201 is configured to display the image 700 producedby the image producing unit 21. The light source device 202 isconfigured to illuminate the image 700 displayed on the liquid crystalpanel 201 to emit rays of light constituting the displayed image 700forward of the liquid crystal panel 201. The display control unit 203 isconfigured to control the liquid crystal panel 201 and the light sourcedevice 202.

The liquid crystal panel 201 includes a Liquid Crystal Display (LCD),for example. The liquid crystal panel 201 is placed in front of thelight source device 202. The liquid crystal panel 201 has a frontsurface (a surface away from the light source device 202) serving as thedisplay surface 20 a. The image 700 is displayed on the display surface20 a.

The light source device 202 serves as a backlight of the liquid crystalpanel 201. An output light of the light source device 202 passes throughthe liquid crystal panel 201 to exit from the display surface 20 a. Thelight source device 202 may include a solid-state light emittingdevice(s) such as a light emitting diode(s), a laser diode(s), and thelike. The light source device 202 may be a surface light sourceconfigured to illuminate a substantially whole area of a back surface ofthe liquid crystal panel 201.

The display control unit 203 is configured to drive the liquid crystalpanel 201 based on the image 700 output from the image producing unit 21to the display unit 20 to make the display surface 20 a display thereonthe image 700. The display control unit 203 is configured to turn on thelight source device 202 to illuminate the image 700 displayed on theliquid crystal panel 201 to emit rays of light constituting the image700 forward. The rays of light emitted forward of the liquid crystalpanel 201 represent the image 700 displayed on the liquid crystal panel201. Accordingly, the image displayed on the liquid crystal panel 201 isprojected forward of the liquid crystal panel 201 with the output lightof the light source device 202.

The display control unit 203 includes a microcomputer including aCentral Processing Unit (CPU) and a memory, for example. In other words,the display control unit 203 is realized by a computer (processor)including the CPU and the memory, and the computer, when the CPUexecutes a computer program stored in the memory, functions as thedisplay control unit 203. The computer program is stored in the memoryof the computer for the display control unit 203, but at least part ofthem can be provided through a telecommunication network such as theInternet or a non-transitory recording medium such as a memory card.

According to the image display system 10 configured as described above,it is possible to correct the display position of the virtual image 300in the target space 400 in response to the orientation change of thebody unit 1 with a less delay time for displaying the image 700corresponding to the virtual image 300 and with a less restriction onthe drawable area for drawing the image 700. Specifically, according tothe image display system 10, the correction processing of correcting thedisplay position of the virtual image 300 in the target space 400according to the orientation change of the body unit 1 includes thefirst correction processing addressing the first orientation change ofthe body unit 1 and the second correction processing addressing thesecond orientation change of the body unit 1 which are performedseparately. Accordingly, the second correction processing required to beperformed in real time basis can be performed in a processing stagecapable of reducing the display delay caused by the second correctionprocessing, while the first correction processing which is notnecessarily performed in real time basis can be performed in anotherprocessing stage in which the drawable area of the image 700 is lesslikely to be restricted by the first correction processing.

(3) Details of the Correction Processing by the Drawing Unit (i.e., theCorrection Processing of the First Correction Unit)

The correction processing by the drawing unit 24 (i.e., the correctionprocessing of the first correction unit 41) will be described in detailwith reference to FIG. 5A to FIG. 6B.

It is explained first is a drawing method of drawing the image 700 bythe drawing unit 24. In the drawing unit 24, the drawing main unit 241is configured to draw the image 700. As shown in FIG. 5A, the drawingmain unit 241 has a three-dimensional virtual space 70 for drawing theimage 700. The three-dimensional virtual space 70 is configured to beset in a storage device provided in the drawing main unit 241.

The three-dimensional virtual space 70 corresponds to the target space400. There are a virtual optical axis 71, a projection plane 72, avirtual mad surface 74, and a predetermined position 75 set in thethree-dimensional virtual space 70. The virtual axis 71, the projectionplane 72, the virtual road surface 74, and the predetermined position 75correspond to the optical axis 500, the imaginary surface 501, the roadsurface 600, and the predetermined position 401 in the target space 400,respectively. The projection plane 72 is substantially perpendicular tothe virtual axis 71.

As shown in FIG. 5A, the drawing main unit 241 is configured to render athree-dimensional image 73 corresponding to the image 700 in thethree-dimensional virtual space 70. The three-dimensional image 73 ispositioned at the predetermined position 75 within the three-dimensionalvirtual space 70 so that a prescribed positional relation between thethree-dimensional image 73 and the virtual road surface 74 (for example,they are parallel to each other) is maintained. The drawing main unit241 is further configured to project the rendered three-dimensionalimage 73 onto the projection plane 72 in the three-dimensional virtualspace 70 to form a two-dimensional image. The resultant two-dimensionalimage serves as the image 700.

In a case where there is no orientation change in the automobile 100,the virtual image 300 corresponding to the image 700 drawn as describedabove is displayed at the predetermined position 401 (the position wherethe arrow overlaps with the road in the intersection), as shown in FIG.5B.

When the first orientation change occurs in the body unit 1, the line ofthe user's 200 sight moves upward for example, as a result the realscene seen by the user 200 moves relatively downward, for example. Thiscauses the virtual image 300 to be displayed at a position above theintersection in the target space 400 (see “300X” in FIG. 6B).

When the first orientation change occurs in the body unit 1 as describedabove, the drawing main unit 241 corrects the display position of thevirtual image 300 by drawing the image 700 so that the misalignment ofthe virtual image 300 with respect to the real scene caused by the firstorientation change of the body unit 1 is compensated. Specifically, thedrawing main unit 241 renders the three-dimensional image 73 within thethree-dimensional virtual space 70 so that the misalignment of thevirtual image 300 with respect to the real scene caused by the firstorientation change of the body unit 1 can be compensated by renderingthe three-dimensional image at a position 76 displaced downward from thepredetermined position 75 (see FIG. 6A). In FIG. 6A, thethree-dimensional image before the correction is designated by areference sign “73X”, and the three-dimensional image after thecorrection is designated by a reference sign “73Y”.

The drawing main unit 241 then projects the rendered (corrected)three-dimensional image 73Y onto the projection plane 72 in thethree-dimensional virtual space 70 to form a two-dimensional image. Theresultant two-dimensional image serves as a corrected image 700. Avirtual image 300Y corresponding to the corrected image 700 is displayedat a position (a position where the arrow overlaps with the road in theintersection) displaced downward from the predetermined position 401 inthe target space 400, as shown in FIG. 6B. In FIG. 6B, the pre-correctedvirtual image is designated by a reference sign “300X”, and thecorrected virtual image is designated by a reference sign “300Y”.

(4) Details of the Correction Processing by the Vibration DistortionCorrection Unit (i.e., the Correction Processing of the SecondCorrection Unit)

The correction processing by the vibration distortion correction unit253 (i.e., the correction processing of the second correction unit 42)will be described in detail with reference to FIG. 7 , FIG. 8A and FIG.8B.

In FIG. 7 , the image and an arrow inside the image before beingsubjected to the vibration correction processing and the distortioncorrection processing are designated by reference signs “700X” and“710X”, respectively. In FIG. 7 , the image and an arrow inside theimage after being subjected to the vibration correction processing andthe distortion correction processing are designated by reference signs“700Y” and “710Y”, respectively.

The vibration distortion correction unit 253 is configured to performthe vibration correction processing on the image 700 output from theoutput buffer 252, and subsequently perform the distortion correctionprocessing on the resultant image. The vibration correction processingand the distortion correction processing are performed sequentially inthis order. The vibration correction processing corrects themisalignment of the virtual image 300 with respect to the real scene,which is caused by the second orientation change of the body unit 1. Thedistortion correction processing corrects the distortion of the virtualimage 300 caused by the reflection of light by the windshield 101 andthe projection unit 3.

As shown in FIG. 7 , the vibration distortion correction unit 253includes a drawing buffer 80 and a line buffer 81.

The drawing buffer 80 is configured to temporarily store the image 700output from the output buffer 252. According to the drawing buffer 80, areadout start point P1 for reading out the image 700 from the drawingbuffer 80, from which an image is read out, can be changed (can be setat a desired point). In an example where the readout start point P1 isset to an upper left corner (a reference point), read out from thedrawing buffer 80 is an image having one screen size and read out fromthe upper left corner as the reference point. In this case, the readoutimage 700 can be read out with no displacement in its layout.

In another example where the readout start point P1 is set to a point P1a which is displaced upward from the upper left corner of the image 700,read out from the drawing buffer 80 is an image having the one screensize but read out from the readout start point P1 as the referencepoint. In this case, the readout image 700 has a layout displaceddownward, as a result of the readout start point displaced upward.Accordingly, the image 700Y is displayed on the display surface 20 a sothat the arrow 710 in the image 700Y is displaced downward. Note that inyet another example where the readout start point P1 is set to a pointdisplaced downward from the upper left corner of the image 700, theimage is displayed on the display surface 20 a so that the arrow 710Y inthe image 700Y is displaced upward.

The image 700 is read out from the drawing buffer 80 on a sub-area image701 basis (one sub-area image by one sub-area image), wherein eachsub-area image 701 is an image area including some pixel lines. Notethat the image 700 having the one screen size is constituted by acollection of the sub-area images 701.

The line buffer 81 includes a storage device configured to store thesub-area image 701 which is an image area including some pixel lines.The line buffer 81 is configured to temporarily store the sub-area image701 read out from the drawing buffer 80, and to allow the sub-area image701 subject to the distortion processing to be output to the displaycontrol unit 203 on a basis of an image having one pixel line (one pixelline image by one pixel line image).

The vibration distortion correction unit 253 is configured totemporarily store the image 700X output from the output buffer 252 inthe drawing buffer 80. The vibration distortion correction unit 253 isconfigured to read out the image 700X stored in the drawing buffer 80from a designated readout start point P1 on the sub-area image 701basis, and to temporarily store the readout sub-area image 701 in theline buffer 81. The vibration distortion correction unit 253 is furtherconfigured to perform the distortion correction processing on thesub-area image 701 stored in the line buffer 81, and read out thesub-area image 701 from the line buffer 81 and output it to the displayunit 20 on a basis of an image having one pixel line, where a collectionof the images having one pixel line constitutes the sub-area image 701.As a result, the image 700Y subjected to the vibration correctionprocessing and the distortion correction processing is displayed in thedisplay unit 20.

The vibration distortion correction unit 253 is configured to detect,based on the second orientation signal from the vibration sensor 5, theangle of pitch of the body unit 1 caused by the second orientationchange, and to change (adjust) the readout start point P1 to compensatethe misalignment of the virtual image 300 with respect to the real sceneaccording to the angle of pitch. In an case where the angle of pitchcorresponds to the backward inclination of the body unit 1, the readoutstart point P1 is changed to the point P1 a positioned above the upperleft corner of the image 700 so that the virtual image 300 is displayedto a position lower than the predetermined position 401 in the targetspace 400. The vibration correction processing is realized by thisprocessing that the image 700 is read out from the changed readout startpoint P1 a. As a result, the display position of the arrow 710Y of thecorrected image 700Y in the display unit 20 is corrected (adjusted) tothe position lower than the display position of the arrow 710X ofpre-corrected image 700X. Accordingly, the misalignment of the virtualimage 300 with respect to the real scene caused by the vibration of thebody unit 1 can be compensated.

It should be noted that, when the readout start point P1 is changed to aposition displaced from the upper left corner of the image 700, thereadout image 700 may include a blank region 700 s having no image dueto displacement in the layout of the image 700. This blank region 700 smay be displayed as a black image, for example. Furthermore, a blankregion having no image may be generated as a result of the distortioncorrection processing in some cases. The blank region 700 t in this casemay also be displayed as a black image.

The vibration distortion correction processing is performed at a stagelater than the output buffer 252 and before the display unit 20, asdescribed above. This can reduce the display delay in the display unit20. According to the distortion correction processing performed on theimage 700, the image 700 is read out from the drawing buffer 80 to theline buffer 81 on the sub-area image 701 basis, and the distortioncorrection processing is performed on the sub-area image 701 basis aswell, and then the resultant image is output to the display unit 20.This can reduce the delay of time for outputting the image from the linebuffer 81 to the display unit 20, and as a result can further reduce thedisplay delay in the display unit 20.

In the vibration correction processing, the correction amount (amount ofmisalignment) H1 for correcting the display position of the image 700may vary non-linearly with respect to the magnitude of the change amount(angle of pitch) α1 in the second orientation change of the body unit 1.In this case, as shown in FIG. 8 , the correction amount H1 variesaccording to the magnitude of the change amount α1. When the changeamount α1 is smaller than a predefined value (namely, when the changeamount α1 is comparatively small), the correction amount H1 may becomparatively small or may be zero (substantially zero). When the changeamount α1 is larger than or equal to a predefined value, the correctionamount H1 may be proportional to the change amount α1. With thisconfiguration, the vibration correction processing may not be performedwhen the change amount α1 according to the second orientation change ofthe body unit 1 is comparatively small, and the vibration correctionprocessing may be formed when the change amount α1 according to thesecond orientation change of the body unit 1 is comparatively large. Forexample, the display position of the virtual image 300 may not becorrected in a case where the body unit 1 installed in the automobile100 vibrates due to idling of an engine. Note that the correction amountH1 may be proportional to the change amount α1 with respect to a wholerange of the change amount α1.

In the example described above, the second orientation change of thebody unit 1 corresponds to the change according to a pitch direction ofthe automobile 100, but is not limited thereto. Additionally oralternatively, the second orientation change of the body unit 1 may be achange according to a yaw direction of the automobile 100. In this case,the readout start point P1 may be changed to a left side or a right sideaccording to the angle of yaw of the automobile 100. As a result, theimage 700 is displayed in the display unit 20 so that the displayposition of the arrow 710 in the corrected image 700 is displaced in theleft side or right side from the position of the arrow 710 in thepre-corrected image 700. Additionally or alternatively, the secondorientation change may be a change according to a roll direction. Inthis case, the image 700 is read out so that it is rotated according toan angle of roll with respect to the readout start point P1.Accordingly, the image 700 is displayed in the display unit 20 so thatthe display position of the arrow 710 in the corrected image 700 isrotatably displaced from the position of the arrow 710 of thepre-corrected image 700. Note that the yaw direction is a directionaround an axis running up and down of the automobile 100, and the angleof yaw is an angle of rotation of the yaw direction. The roll directionis a direction around an axis running from the front to the back of theautomobile 100, and the angle of roll is an angle of rotation of theroll direction.

In the explanation described above, the distortion correction processingis performed on the sub-area image 701 basis, but is not limitedthereto. For example, the distortion correction processing may beperformed on the image 700 basis. In this case, the line buffer 81 maybe replaced with a drawing buffer capable of temporarily storing theimage having one screen size, and the distortion correction processingmay be performed on the drawing buffer.

(5) Variations

The above embodiment is merely one example of various embodiments of thepresent disclosure. The above embodiment may be modified in various waysin accordance with design or the like as it can achieve the object ofthe present disclosure. Furthermore, aspects of the above embodiment arenot limited to be realized in a form of the image display system alone.For example, some aspects of the above embodiment may be realized in aform of a movable object including the image display system 10, an imagedisplay method employing the image display system 10, or the like. Someaspects of the above embodiment may be realized in a form of a computerprogram which when executed by a computer causes the computer to performthe above image display method, a non-transitory recording mediumstoring the above computer program, or the like. Variations and theabove embodiment can be appropriately combined.

In the above embodiment, a single vibration sensor 5 is used to detectboth of the first orientation change and the second orientation changeof the body unit 1, and the two filters 6 and 7 are used to distinguishthe first orientation change and the second orientation change togenerate individual orientation signal, but the present embodiment isnot limited thereto. For example, the vibration sensor 5 may be replacedwith two sensors, a first vibration sensor (first detection unit) and asecond vibration sensor (second detection unit). The first vibrationsensor is configured to detect the first orientation change of the bodyunit 1 and to output the first orientation signal indicative of thedetected first orientation change. The second vibration sensor isconfigured to detect the second orientation change of the body unit 1and to output the second orientation signal indicative of the detectedsecond orientation change. In this case, the low-pass filter 6 and thehigh-pass filter 7 may be omitted.

The projection unit 3 is not particularly limited to as long as itincludes at least an optical element. The projection unit 3 is notlimited to include the two mirrors, the first mirror 31 and the secondmirror 32, but may include a single mirror alone or three or moremirrors. The projection unit 3 may include an optical element other thanthe mirror, such as a lens.

The image display system 10 is not limited to form the virtual image 300in the target space 400 set in front of the automobile 100 according tothe traveling direction, but may be configured to form the virtual image300 in a back of, in a side of, or above the automobile 100 according tothe traveling direction.

The image display system 10 is not limited to be employed as the head-updisplay for the automobile 100, but may be employed for another movableobject body other than the automobile 100, such as a motorcycle, atrain, an air-plane, a construction machinery, a ship or a boat, and thelike. The image display system 10 is not limited to be employed in themovable object, but may be employed in an amusement machine or the like,for example. The image display system 10 may also be employed as awearable appliance such as a Head Mounted Display (HMD), a medicalmachine, a stationary device, or the like. The image display system 10may be incorporated in a device such as a digital camera to function asan electronic viewfinder.

In the present embodiment, the drawing unit 24, the correction unit 25,and the display control unit 203 are realized by individual CPUs andmemories, but the drawing unit 24, the correction unit 25, and thedisplay control unit 203 may be realized by a single CPU and a singlememory. Any two of the drawing unit 24, the correction unit 25, and thedisplay control unit 203 may be realized by a single CPU and a singlememory.

The drawing unit 24 may be realized by a Graphics Processing Unit (GPU)in place of the CPU. The correction unit 25 may be realized by aField-Programmable Gate Array (FPGA) in place of the CPU.

In the present embodiment, the vibration distortion correction unit 253(i.e., the correction unit 25) serves as the correction unit configuredto perform the correction processing addressing the second orientationchange of the body unit 1, but is not limited thereto. For example, thevibration distortion correction unit 253 may serve as a correction unitconfigured to perform a correction processing addressing a wholeorientation change of the body unit 1 (in other words, an orientationchange not separated to the first orientation change and the secondorientation change).

In the above embodiment, the display unit 20 includes the liquid crystalpanel 201, but is not limited thereto. Alternatively, a display unit 20may be configured to form an image 700 by scanning a display surface 20a with a laser beam striking from a back of the display surface 20 a ofthe display unit 20, as shown in FIG. 9 .

Specifically, the display unit 20 includes a diffusively transmissivescreen 90, a light emitter 91 configured to emit light to the screen 90from a back side of the screen 90. The light emitter 91 is a scanningtype light emitting unit, and is configured to emit light K1 to thescreen 90. The front surface or the back surface (the front surface, inthis example) of the screen 90 serves as the display surface 20 a, andthe image 700 is drawn by the light K1 emitted from the light emitter91. As a result, the virtual image 300 (see FIG. 2 ) is formed in thetarget space 400 with the light K1 passing through the screen 90.

The light emitter 91 includes a light source 911 configured to producethe light (laser beam, for example) K1, a scanning unit 912 for scanningof the light K1 of the light source 911, and a lens 913. The lightsource 911 includes a laser module configured to emit the light K1. Thescanning unit 912 is configured to reflect the light K1 of the lightsource 911 toward the screen 90 through the lens 913. The scanning unit912 is configured to change the direction of the reflection of the lightK1, and to thereby cause the light K1 to run on the display surface 20 aof the screen 90 to scan the display surface 20 a. The scanning unit 912can realize a Raster Scan of two-dimensional scanning with the light K1horizontally and vertically on the display surface 20 a. The scanningunit 912 is configured to scan the display surface 20 a by moving abright spot thereon to form a two-dimensional image (the image 700, forexample) on the display surface 20 a. The bright spot may be a point atwhich the ray of light K1 crosses with the display surface 20 a of thescreen 90. The scanning unit 912 includes a micro scanning mirrormanufactured by Micro Electro Mechanical Systems (MEMS) techniques, forexample. The scanning unit 912 includes a rotatable optical element(mirror) for reflecting the light K1, and reflects the light K1 of thelight source 911 toward a direction according to a rotation angle(deflection angle) of the optical element. The scanning unit 912 scansthe light K1 of the light source 911 accordingly. The scanning unit 912realizes the Raster Scan in which scanning is performedtwo-dimensionally with the light K1 by rotating the optical elementabout two axes perpendicular to each other, for example.

Aspects

An image display system (10) of a first aspect includes an imageproducing unit (21), a display unit (22), a projection unit (3), and abody unit (1). The image producing unit (21) is configured to produce animage (700). The display unit (20) is configured to display the image(700) produced by the image producing unit (21). The projection unit (3)is configured to project, in a target space (400), a virtual image (300)corresponding to the image (700) with an output light of the displayunit (20). The display unit (22) and the projection unit (3) areprovided to the body unit (1). The image producing unit (21) includes afirst correction unit (41) and a second correction unit (42). The firstcorrection unit (41) is configured to perform a first correctionprocessing of correcting, based on a first orientation signal, a displayposition of the virtual image (300) in the target space (400). The firstorientation signal is indicative of a first orientation change of thebody unit (1). The second correction unit (42) is configured to performa second correction processing of correcting, based on a secondorientation signal, the display position of the virtual image (300) inthe target space (400). The second orientation signal is indicative of asecond orientation change of the body unit (1). A change rate of thesecond orientation change is faster than that of the first orientationchange. The second correction processing is performed at a timingdifferent from a timing at which the first correction processing isperformed.

Preferably, the image display system is installed in a movable object.Preferably, the body unit exhibits an orientation change in response toa vibration (an acceleration) acting on the movable object, theorientation change including a first orientation change and a secondorientation change of which change rate is faster than that of the firstorientation change.

With this configuration, it is possible to correct the display positionof the virtual image (300) in the target space (400) in response to theorientation change of the body unit (1) with a less delay time fordisplaying the image (700) corresponding to the virtual image (300) andwith a less restriction on the drawable area for drawing the image(700). Specifically, the correction processing of correcting the displayposition of the virtual image (300) in the target space (400) accordingto the orientation change of the body unit (1) includes a firstcorrection processing and a second correction processing which areperformed separately. The first correction processing addresses thefirst orientation change of the body unit (1). The second correctionprocessing addresses the second orientation change of the body unit (1).A change rate of the second orientation change is faster than that ofthe first orientation change. The second correction processing isperformed at a timing different from a timing at which the firstcorrection processing is performed by the first correction unit (41).Accordingly, the second correction processing required to be performedin real time basis can be performed in a processing stage capable ofreducing the display delay caused by the second correction processing.The first correction processing which is not necessarily performed inreal time basis can be performed in another processing stage in whichthe drawable area of the image 700 is less likely to be restricted bythe first correction processing.

In an image display system (10) of a second aspect, which may berealized in combination with the first aspect, the second correctionunit (42) is configured to perform, on the image (700), a distortioncorrection processing in addition to the second correction processing.

With this configuration, the second correction unit (42) also serves asa correction unit for performing the distortion correction processing.It is accordingly possible to perform the second correction processingand the distortion correction processing together, leading toimprovement of processing efficiency.

In an image display system (10) of a third aspect, which may be realizedin combination with the first or second aspect, the second correctionunit (42) includes a buffer (252) and a correction main unit (253). Thebuffer (252) is configured to store therein the image (700). Thecorrection main unit (253) is configured to perform the secondcorrection processing on the image (700) stored in the buffer (252), andto output, to the display unit (20), the image (700) on which the secondcorrection processing is performed.

With this configuration, the second correction processing is performedat a stage later than the buffer (252) and before the display unit (20).This can further reduce the display delay in the display unit (20)caused by the second correction unit.

An image display system (10) of a fourth aspect, which may be realizedin combination with any one of the first to third aspects, furtherincludes a first detection unit (5, 6) and a second detection unit (5,7). The first detection unit (5, 6) is configured to output the firstorientation signal. The second detection unit (5, 7) is configured tooutput the second orientation signal.

With this configuration, it is possible to provide an image displaysystem including the first detection unit (5, 6) configured to outputthe first orientation signal and the second detection unit (5, 7)configured to output the second orientation signal.

In an image display system (10) of a fifth aspect, which may be realizedin combination with the second aspect, the distortion correctionprocessing is performed after the second correction processing isperformed.

In an image display system (10) of a sixth aspect, which may be realizedin combination with the third aspect, the correction main unit (253)includes a drawing buffer (80) and a line buffer (81). The drawingbuffer (80) is configured to store therein the image (700) which has onescreen size and is output from the buffer (252). The line buffer (81) isconfigured to store therein an image read out from the drawing buffer(80). The drawing buffer (80) is configured such that, when the image(700) stored in the drawing buffer is read out therefrom, an imagehaving the one screen size and defined by a readout start point is readout on a sub-area image (701) basis, wherein a collection of a pluralityof the sub-area images (701) constitutes the image having the one screensize. The line buffer (81) is configured to store the sub-area image(701) read out from the drawing buffer (80), and to allow the sub-areaimage (701) on which a distortion processing is performed to be outputto the display unit (20) on one pixel line basis. The drawing buffer(80) is configured to detect, based on the second orientation signal,the second orientation change of the body unit (1), and to change thereadout start point so as to compensate a misalignment of the virtualimage with respect to a real scene by the detected second orientationchange, to perform the second correction processing.

An image display system (10) of a seventh aspect, which may be realizedin combination with the fourth aspect, includes a vibration sensor (5),a low-pass filter (6), and a high-pass filter (7). The vibration sensor(5) is configured to detect a vibration acting on the body unit (1). Thelow-pass filter (6) is configured to allow a signal component having afrequency lower than the first frequency, of a signal output from thevibration sensor (5), to pass therethrough, and not to allow a signalcomponent having a frequency higher than or equal to a first frequency,of the signal, to pass. The high-pass filter (7) is configured to allowa signal component having a frequency higher than or equal to the secondfrequency, of the signal output from the vibration sensor (5), to passtherethrough, and not to allow a signal component having a frequencylower than a second frequency, of the signal to pass. The firstdetection unit (5, 6) includes the vibration sensor (5) and the low-passfilter (6). The second detection unit (5, 7) includes the vibrationsensor (5) and the high-pass filter (7).

In an image display system (10) of an eighth aspect, which may berealized in combination with the first aspect, the image producing unit(21) includes a drawing main unit (241). The drawing main unit (241) isconfigured to produce the image (700), and correct the image (700) basedon the first orientation signal to compensate a misalignment of thevirtual image (300) with respect to a real scene, which is caused by thefirst orientation change of the body unit (1). The drawing main unit(241) includes the first correction unit (41).

In an image display system (10) of a ninth aspect, which may be realizedin combination with the eighth aspect, the drawing main unit (241)includes a storage device in which a three-dimensional virtual space(70) is set. The three-dimensional virtual space (70) corresponds to thetarget space (400). The drawing main unit (241) is configured to producethe image (700) by rendering a three-dimensional image (73)corresponding to the image (700) within the three-dimensional virtualspace (70) and projecting the three-dimensional image (73) onto aprojection plane (72) in the three-dimensional virtual space (70). Thedrawing main unit (241) is configured to, when the first orientationchange occurs in the body unit (1), correct the display position of thevirtual image (300) by correcting the three-dimensional image (73) tocompensate a misalignment of the virtual image (300) with respect to areal scene, which is caused by the first orientation change of the bodyunit (1).

In an image display system (10) of an tenth aspect, which may berealized in combination with the first aspect, the display unit (20) isconfigured to scan the display surface (21 a) with a laser beam to formthe image (700), the laser beam striking from a back of the displaysurface (20 a) of the display unit (20).

In an image display system (10) of an eleventh aspect, which may berealized in combination with the tenth aspect, the display unit (20)includes a screen (90), and a light emitter (91). The light emitter (91)is configured to emit light of the laser beam to the screen (90) toproject the virtual image (300) toward the target space (400) with thelight (K1) passing through the screen (90).

In an image display system (10) of a twelfth aspect, which may berealized in combination with the first aspect, the second correctionunit (42) is configured to, in the second correction processing,determine a correction amount for correcting a display position of theimage (700) produced by the image producing unit (21), and thecorrection amount varies non-linearly according to a magnitude of achange amount (α1) of the second orientation change of the body unit(1).

In an image display system (10) of a thirteenth aspect, which may berealized in combination with the twelfth aspect, the second correctionunit (42) is configured to determine the correction amount (H1) tosubstantially zero when the change amount (α1) is smaller than apredefined value, and the correction amount (H1) varies linearlyaccording to the change amount (α1) larger than or equal to a predefinedvalue.

In an image display system (10) of a fourteenth aspect, which may berealized in combination with the twelfth aspect, the second orientationchange includes a change in an orientation of the body unit (1)according to a pitch direction, a yaw direction, or a roll direction ofthe body unit (1).

A movable object of a fifteenth aspect includes the image display system(10) of any one of the first to fourteenth aspects, and a movable objectbody (100) in which the image display system (10) is installed.

In a movable object of a sixteenth aspect, which may be realized incombination with the fifteenth aspect, the projection unit (3) isconfigured to project the image (700) displayed in the display unit (20)onto a windshield (101) of the movable object body (100) to form thevirtual image (300) in the target space (400).

In a movable object of a seventeenth aspect, which may be realized incombination with the fifteenth aspect, the virtual image (300) indicatesa driving assist information for assisting driving the movable objectbody (100). The driving assist information includes at least one ofvehicle speed information, navigation information, pedestrianinformation, forward vehicle information, lane departure information,and vehicle condition information.

In a movable object of an eighteenth aspect, which may be realized incombination with the fifteenth aspect, the movable object body (100) isan automobile, a motorcycle, a train, a construction machinery, a shipor a boat.

An image display method of nineteenth aspect employs an image displaysystem including a display unit (20), a projection unit (3), and a bodyunit (1). The display unit (20) is configured to display an image (700).The projection unit (3) is configured to project in a target space (400)a virtual image (300) corresponding to the image (700) with an outputlight of the display unit (20). The display unit (20) and the projectionunit (3) are provided to the body unit (1). The image display methodincludes an image producing processing of producing the image (700)displayed on the display unit (20). The image producing processingincludes a first correction processing and a second correctionprocessing. The first correction processing is a processing ofcorrecting, based on a first orientation signal, a display position ofthe virtual image (300) in the target space (400). The first orientationsignal is indicative of a first orientation change of the body unit (1).The second correction processing is a processing of correcting, based ona second orientation signal, the display position of the virtual image(300) in the target space (400). The second orientation signal isindicative of a second orientation change of the body unit (1). A changerate of the second orientation change is faster than that of the firstorientation change. The image display method performs the firstcorrection processing and the second correction processing at differenttimings.

With this configuration, it is possible to correct the display positionof the virtual image (300) in the target space (400) in response to theorientation change of the body unit (1) with a less delay time fordisplaying the image (700) corresponding to the virtual image (300) andwith a less restriction on the drawable area for drawing the image(700). Specifically, the correction processing of correcting the displayposition of the virtual image (300) in the target space (400) accordingto the orientation change of the body unit (1) includes a firstcorrection processing and a second correction processing which areperformed separately. The first correction processing addresses thefirst orientation change of the body unit (1). The second correctionprocessing addresses the second orientation change of the body unit (1).A change rate of the second orientation change is faster than that ofthe first orientation change. The second correction processing isperformed at a timing different from a timing at which the firstcorrection processing is performed by the first correction unit (41).Accordingly, the second correction processing required to be performedin real time basis can be performed in a processing stage capable ofreducing the display delay caused by the second correction processing.The first correction processing which is not necessarily performed inreal time basis can be performed in another processing stage in whichthe drawable area of the image 700 is less likely to be restricted bythe first correction processing.

A non-transitory computer-readable medium of twentieth aspect records acomputer program for instructing a computer system to execute an imagedisplay method employing an image display system (10). The image displaysystem (10) includes an image display unit (20), a projection unit (3),and a body unit (1). The display unit (20) is configured to display animage (700). The projection unit (3) is configured to project in atarget space (400) a virtual image (300) corresponding to the image(700) with an output light of the display unit (20). The display unit(20) and the projection unit (3) are provided to the body unit (1). Theimage display method includes an image producing processing of producingthe image (700) displayed on the display unit (20). The image producingprocessing includes a first correction processing and a secondcorrection processing. The first correction processing is a processingof correcting, based on a first orientation signal, a display positionof the virtual image (300) in the target space (400). The firstorientation signal is indicative of a first orientation change of thebody unit (1). The second correction processing is a processing ofcorrecting, based on a second orientation signal, the display positionof the virtual image (300) in the target space (400). The secondorientation signal is indicative of a second orientation change of thebody unit (1). A change rate or the second orientation change is fasterthan that of the first orientation change. The image display methodperforms the first correction processing and the second correctionprocessing at different timings.

The computer-readable medium with this configuration allows a generalcomputer system to correct the display position of the virtual image(300) in the target space (400) in response to the orientation change ofthe body unit (1) with a less delay time for displaying the image (700)corresponding to the virtual image (300) and with a less restriction onthe drawable area for drawing the image (700).

While various embodiments have been described herein above, it is to beappreciated that various changes in form and detail may be made withoutdeparting from the spirit and scope of the present disclosure presentlyor hereafter claimed.

The entire contents of Japanese Patent Application No. 2018-053557mentioned above are incorporated by reference.

What is claimed is:
 1. An image display system, comprising: an imagegenerator that produces an image corresponding to a virtual imageprojected in a target space; a display that displays the image producedby the image generator, and a body to which the display is provided,wherein: the image generator includes a first corrector that performs afirst correction processing of correcting a display position of thevirtual image in the target space, based on a first orientation signal,the image generator includes a main drawer that produces the image andcorrect the image based on the first orientation signal to compensate amisalignment of the virtual image with respect to a real scene, which iscaused by the first orientation change of the body, and the main drawerincludes the first corrector.
 2. The image display system of claim 1,wherein the image generator further includes a second corrector thatperforms a second correction processing of correcting the displayposition of the virtual image in the target space, based on a secondorientation signal, the second orientation signal is indicative of asecond orientation change of the body of which change rate is fasterthan that of the first orientation change, and the second correctorperforms the second correction processing at a timing different from atiming at which the first corrector performs the first correctionprocessing.
 3. The image display system of claim 2, wherein the secondcorrector performs, on the image, a distortion correction processing inaddition to the second correction processing.
 4. The image displaysystem of claim 3, wherein the distortion correction processing isperformed after the second correction processing is performed.
 5. Theimage display system of claim 2, wherein the second corrector includes:a buffer that stores therein the image; and a main corrector thatperforms the second correction processing on the image stored in thebuffer, and to output, to the display, the image on which the secondcorrection processing is performed.
 6. The image display system of claim5, wherein the main corrector includes: a drawing buffer that storestherein the image which has one screen size and is output from thebuffer; and a line buffer that stores therein an image read out from thedrawing buffer, when the image stored in the drawing buffer is read outtherefrom, an image having the one screen size and defined by a readoutstart point is read out on a sub-area image basis, and a collection of aplurality of the sub-area images constitutes the image having the onescreen size, the line buffer stores the sub-area image read out from thedrawing buffer, and provides for the sub-area image on which adistortion processing is performed to be output to the display on onepixel line basis, and the drawing buffer detects the second orientationchange of the body based on the second orientation signal, and changesthe readout start point so as to compensate a misalignment of thevirtual image with respect to areal scene by the detected secondorientation change, to perform the second correction processing.
 7. Theimage display system of claim 1, further comprising: a first detectorthat outputs the first orientation signal; and a second detector thatoutputs the second orientation signal.
 8. The image display system ofclaim 7, further comprising: a vibration sensor that detects a vibrationacting on the body, a low-pass filter which: allows a signal componenthaving a frequency lower than a first frequency, of a signal output fromthe vibration sensor, to pass therethrough, and does not allow a signalcomponent having a frequency higher than or equal to the firstfrequency, of the signal, to pass; and a high-pass filter which: allowsa signal component having a frequency higher than or equal to a secondfrequency, of the signal output from the vibration sensor, to passtherethrough, and does not allow a signal component having a frequencylower than the second frequency, of the signal, to pass, wherein: thefirst detector includes the vibration sensor and the low-pass filter,and the second detector includes the vibration sensor and the high-passfilter.
 9. The image display system of claim 1, wherein the main drawerincludes a storage device in which a three-dimensional virtual spacecorresponding to the target space is set, and the main drawer: producesthe image by rendering a three-dimensional image corresponding to theimage within the three-dimensional virtual space and projecting thethree-dimensional image onto a projection plane in the three-dimensionalvirtual space, and, when the first orientation change occurs in thebody, corrects the display position of the virtual image by correctingthe three-dimensional image to compensate a misalignment of the virtualimage with respect to a real scene, which is caused by the firstorientation change of the body.
 10. The image display system of claim 1,wherein the display scans the display surface with a laser beam to formthe image, and the laser beam strikes from a back of the display surfaceof the display.
 11. The image display system of claim 10, wherein thedisplay includes: a screen; and a light emitter that emits light of thelaser beam to the screen to project the virtual image toward the targetspace with the light passing through the screen.
 12. The image displaysystem of claim 1, wherein the second corrector, in the secondcorrection processing, determines a correction amount for correcting adisplay position of the image produced by the image generator, and thecorrection amount varies non-linearly according to a magnitude of achange amount of the second orientation change of the body.
 13. Theimage display system of claim 12, wherein the second correctordetermines the correction amount to substantially zero when the changeamount is smaller than a predefined value, and the correction amountvaries linearly according to the change amount larger than or equal to apredefined value.
 14. The image display system of claim 12, wherein thesecond orientation change includes a change in an orientation of thebody according to a pitch direction, a yaw direction, or a rolldirection of the body.
 15. A movable object, comprising: the imagedisplay system of claim 1; and a movable object body in which the imagedisplay system is installed.
 16. The movable object of claim 15, furthercomprising a projector that projects the image displayed in the displayonto a windshield of the movable object body to form the virtual imagein the target space.
 17. The movable object of claim 15, wherein thevirtual image indicates a driving assist information for assistingdriving the movable object body, and the driving assist informationincludes at least one of vehicle speed information, navigationinformation, pedestrian information, forward vehicle information, lanedeparture information, and vehicle condition information.
 18. Themovable object of claim 15, wherein the movable object body is anautomobile, a motorcycle, a train, a construction machinery, a ship or aboat.